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. ABSTRACT 

The purpose of the study was to investigate the 
magnitude of individual diffetences in tire^^to-criterion and the * 
stability of these differences* Tiae-to-cr iter ion was defined in two 
ways: the aaount of elapsed time required to attain the criterion 
level and the amount of on-task time required to attain the criterion 
level. 'Ninety students were randomly assignee to either a mastery 
learning strategy in which all students wer^ tjelped to attain the 85 
percent criterion level or to one of two control classes « All 
students learned a three-^unit sequence of programmed material in 
matriif arithrietic. The results of the study indicated that 
time-on*task-to-criterion and elapsed time-to-criterion are , alterable 
to the extent thott the ratio of the necessary tiae-on-to*criterion 
for the fastest student to the slowest student on the final unit was 
approximately one to one and tvo-^fif ths.. Implications for schooling 
and school learning are discussed. (Author) 
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Abstract 



■ The purpose of the stucty was to investigate the magnitude of 
indiyidual differences in time-to-criterion and the stability of these 
differences. Time-to-criterion was defined in two ways: the amount of 
elapsed time required to attain the criterion level and the amount of 
on-task time required to attain the criterion level . Ninety students 
were randomly assigned to either a mastery learning strategy in which 
all students were helped to attain the 85 per cent criterion level or - 
to one of two control classes. All students learned a three-unit 
sequence of programmed material in matrix arithmetic. The results of 
the study indicated that time-on-task-to-criterion and elapsed time-to- 
cnterion are alterable to the extent that the ratio of the necessary 
time-on-task-to-criterion for the fastest student to the slowest student 
on the final unit was approximately one to one and two-fifths. Implications 
for schooling and school learning are discussed. 
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Time to Criterion: An Experimental Study 

Introduction 

Recent innovations in both instructional design and educational 
measurement have led to a juxtaposition of two concepts in school learning: 
time and amount learned. In contrast to the tendency to set a fixed amount 
of time per learning task, alternative learning strategies (such as Bloom's 
"mastery learning" and Individually Prescribed Instruction) have set fixed 
achievement criteria and provided students with varying amounts of time 
and help to permit virtually all students to attain mastery of these 
educational objectives. 

What 5eems to be involved in this juxtaposition is a trade-off of 
variables. Whereas in the more conventional strategies the variable is 
achievement, the variable in these alternative strategies^ is time. The 
purpose of this study is to examine more closely the nature of this trade- 
off. One commonly held notion is that a certain amount of student variation 
in achievement is exchanged for an equal amount of student variation in 
time-to-criterion and that this student variation in time-to-criterion 
is a relatively stable variation. This is bhe belief that underlies one 
critic's statement that an alternative strategy with fixed achievement 
objectives for all students is not feasible since "it would extend the 
education of some youngsters until they are oldsters" (Cronbach, 1967, p. 25). 

As one examines the. nature of time-to-criterion more closely one sees 
that there are two conceptions of time which can be differentiated. During 
the period of time that is allotted to the student for learning a particular 



task, the student spends a portion of that time working on learning and 
a portion of that time doing things which are not relevant to the task. 
The amount of time that the student is in the presence of the learning 
task can be called "elapsed time,'* The amount of el=^psed time can be 
divided into two parts: time-on-task (the amount of time the student is 
involved in learning) and time-off -task (the amount of time the student 
is not involved in learning)- With these definitions and distinctions 
in mind it is possible to further explore the variable time-to-criterion 
in school learning. 

The main purpose of the study is to investigate the following 
hypothesis. If students are helped to learn the earlier learning units 
in a three unit sequence to a pre-set criterion level, they will spend 
approximately the same of amount of time-on-task to attain the criterion 
level 00 a final unit, despite student difference- tn general ability. 
That is to say, it is hypothesized that stude;tt differences in time-on-task 
to criterion are alterable and can be minimized over a sequence of 
learning units given appropriate learning strategies. 
Prior Research 

Relatively few studies have conducted to investigate time-to-cri terion ^ 
as a variable in school learning- The studies which have been conducted 
fall into one of two categories: those that investigate the magnitude of 
student variation in time-to-criterion and those that investigate the stability 
of time-to-criterion over a series of learning tasks. 

Carroll and Spearritt (1967) examined 96 sixth grade students learning 
a single programmed unit of imaginary second language (MIDIMO)- The sample 
was stratified with respect to intelligence: high, medium, and low- The 



criterion level was set 100 per cent correct. They found a range of 
time-to-criterion of one to four, i.e., it took the slowest student four 
times as long as the fastest student to reach the criterion. 

Block (1970) investigate fourteen eighth grade students learning a 
three unit sequence of matrix arithmetic. The criterion level was set 
at 85 per cent on a content-referenced test. The range of time-to-criterion 
on the initial unit was approximately 1 to 3.4. 

Arlin (1973) studied 37 eleventh grade students learning a seven 
unit sequence of imaginary science. The criterion level was set at 85 
per cent correct. The range of time-to-criterion for the initial unit was 
approximately 1 to 7. 

In combination these studies lend support to Carroll's (1970) estimate 
that the range of time-to-criterion in school learning research is 
approximately one to five. 

The time-to-criterion in the above studies can best be described as 
elapsed time-to-criterion. Time-to-criterion was measured by having the 
students write down the clock time they had begun to learn and the clock 
time they had completed the learning, i.e., the time that they were in 
the presence of the instructional material. 

Only two studies were found that investigated the alterability of 
time-to-criterion over several successive units. One possible reason for 
this lack of empirical research seems to be the tendency for researchers 
to view learning units in isolation in contrast to viewing a single learning 
unit in the context of a series of learning units. Since very few units 
in school learning can be thought of as being isolates there seems, to be 
a need to examine a series or sequence of discrete learning units and 
the consequences of learning (or not learning) each unit on the learning 
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of successive units. 

Both studies which were located did conclude that the range of elapsed 
time-to-criterion decreased over a series of related units when virtually 
all students were helped to attain the criterion level on prior units. 
Block (1970) found a decrease over a three unit sequence from 1 to 3.4 to 
1 to 2.1. Note, however, that Block had a very small sample (n = 14). 
Arlin (1973) found a decrease over a seven unit sequence from 1 to 7 to 
1 to 4. Further study of this problem seems appropriate, especial ly when 
time-on-task-to-criterion is substituted for elapsed time-to-criterion. 
Sampl e 

The subjects were taken from a middle-class suburb of a Midweot»?rn 
community. Lorge-Thorndike verbal intelligence scores ranged from 87 uo 
135. Ninety eighth grade students were randomly assigned to one of three 
classes of thirty students each. Because of absences 26, 27, and 29 
students completed the entire experiment. 
Procedures 

The material to be learned consisted of a three unit sequence of 
programmed material in matrix arithmetic (Block, 1970). The first unit 
was composed of the basic terminology of matrices, the second unit contained 
material on special types of matrices, and the third unit consisted of 
material on matrix operations (addition and subtraction). Eighty minutes 
of class time were allotted for each of the units. 

The three classes learned under two different strategies. The first 
class used mastery learning techniques while the other classes were termed 
control classes. The students in the mastery learning class were given 
additional time and help to attain a pre-set 85 per cent criterion level 
(85ML). The students in one of the control classes were not given the 



additional time and help nor were they required to attain a pre-set 
criterion level (CI), The students in the other .control class were given 
enough additional time and help to reach a 75 per cent criterion level • 
A matrix arithmetic pre-test was administered on the first day of the 
.study. The mean per cent correct for the three classes on pre-test 
was 7.6, 8.9, and 7.9, respectively. The difference among the classes 
was not significant. 

All students were given the programmed text and a unit formative test. 
The students were instructed to write the time on the clock on their 
programmed booklets and begin working. After the students finished the book 
let, they wrote the time they finished and picked up the formative test. 
They wrote the time they began on the formative test, took the fon.iative 
test, and wrote the time they completed the test. All three classes 
proceeded in the same manner up to this point. 

In the mastery learning class {85ML) a student-tutor corrected the 
formative tests as they were completed by the students. Students who did 
not attain the 85 per cent level were asked to complete review exercises 
based on the items that were incorrect. The number of review exercises to 
be completed was determined by the difference between the number of items 
correct on the formative test and the criterion level of seventeen correct 
out of twenty items. The student wrote the time he began working on the 
review exercises and the time he finished. He was then given a parallel 
form of the formative test (a review test) and was to to answer the items 
which were keyed to the particular review exercises performed. Once again 
the student wrote the time he began the test and the time he finished. 
If the student still did not attain the specified criterion level (combining 
the number correct on the formative test with the additioial correct on 
the review test), he repeated the review-corrective process a third time 



with student- tutorisl help and a parallel set of test items • The tutor 
wrote on the student's paper the time he began tuto/ing the student and 
the time he completed the tutoring- Every student in the mastery class 
was able to attain the 85 per cent criterion level with no more than two 
review-corrective periods. The students in the second control group were 
given only one corrective period and were allowed to attain only the 75 
per cent level on the combination of formative tests and review tests on 
the units. The students in the first control group were finished for the 
class period after they had taken the formative test. 

The same procedure was followed for each of the three learning units 
in the sequence. 
Variables 

There were three major variables in the study: elapsed time, time-on- 
task and achievement. Elapsed time refers to the amount of clock time 
that passed between the beginning and end of the actual learning time. 
It was calculated for each student by subtracting the starting time from 
the completion time in every learning segment, i.e., the original learning 
time plus any additional review learning time, and combining the various 
segments. No testing time was included in the computation of elapsed time. 

Time-on-task refers to the time during which the student is actively 
involved in learning. Two types of behaviors are relevant to the student's 
active involvement in learning. The student may be engaged in on-task 
behaviors (e.g., writing) that are, in fact, observable, or he may be engaged 
in on-task behaviors that are unobservable (e.g., thinking). The instrument 
used to estimate the per cent of time-on-task, therefore, included two 



components in an attempt to obtain measurements on both types of 
on-task behaviors. 

The first component was a classroom observation instrument* An 
observer watched a randomly pre-selected student for six seconds and 
coded his behavior as on-task or off-task. He then watched the seco* d 
student in the row for a six second period, coded his behavior, and looked 
to the third stucly. This same procedure was followed until the entire 
class was appropriately coded* The observer then took a second coding ' 
sheet, began with a randomly pre-selected student, and repeated the 
above procedure. This procedure was continued until the end of the class 
period* As a check on the objectivity of the classroom observation 
instrument, a second observer was present in each class for a period of 
30 minutes during the study. The inter-observer agreement for the 
observers on the three on-task and one off-task categories was 83 per cent; 
87 per cent, 75 per cent, and 82 per cent. (See Anderson, 1973, for a 
description of the categories.) The per cent of overt time-on-t^sk for 
each student was estimated by dividing the number of on-task observations 
by the total number of observations. 

The second component of the time-on-task instrument was based on a 
stimulated recall technique developed by Bloom (1953). After every two 
scans of the classroom by the observer, the students were asked to 
stop working and write in a sentence or two what they were thinking just 
prior to being told to stop. The students* thoughts were then classified 
by two judges as being task relevant or task irrelevant. The inter- judge 
agreement was 89 per cent. The per cent of covert time-on-task was 
estimated by dividing the number of thoughts classified by the j^udges as 
task relevant by the total number of thoughts classified. 

10 



The per cent of time-on-task was computeid by taking the arithmetic 
average of the per cent of covert and overt time-on-task. Finally, the 
amount of time-on-task was computed by multiplying the per cent of time- 
o?i-task by the amount of elapsed time. 

Achievement on each unit was defined as the total number correct 
on a combination of the formative test and the review tests. For tha 
purpose of testing the hypothesis, sixteen of twenty items correct, or 
eighty per centv was designated ^s the criterion level. Eighty per cent 
was used as the criterion level in order to produce samples wnich were 
large enough to be meaningful. 

The formative tests were composed of 20 items whidurequi red the 
learner to supply the correct answer. The items were based on a unit 
table of specifications table. The items tested the taxonomic levels of 
knowledge, comprehension, and application (Bloom, 1956). 
Results 

The hypothesis was investigated in two ways. First, the amount of 
time-on-task necessary to reach or surpass the criterion level by three 
different groups of learners; on each of the three learning tasks was 
examined. These three groups were chosen in the followiing manner. Group 1 
consisted of students in the two control classes who attained scores at 
or above the criterion on the formative test t^ken after the original 
amount of initial elapsed time. In other words, these students did not 
need nor receive additional time or'help on the unit. Group 2 consisted 
of students in the 85ML class who yere able to attain the criterion score 
or higher on the specific unit's formative test. Group 3 was composed of 
students in the 85ML class who required additional elapsed time on the 
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particular learning unit* .Some of these students may have attained the 
criterion level in original amounts of learning time in oicher learning 
units* It is important to note that the groups are not invariant across 
all learning units. In other words, for each unit a ne\; Group 1, Group 
2, and Group 3 were formed based on the students who attained the criteric 
level on that particular unit. 

In fact, if we examine Groups 2 and 3, both of whom were in the 85ML 
class, some interesting findings emerge. Of those six students who 
were able to attain the criterion level in the original elapsed time 
5n the first unit only three were able to ?ittain the criterion score in 
original elapsed time on unit two. Further^ only two students were able 
to attain mastery on all three units without extra time and help. On the 
oth*ir hand, twelve of the students in the 85ML class required additional 
help on all three learning units, although as shall be seen, not as m'^ch 
extra time and help in the third unit as in the first unit- 
Three groups were used in the analysis because any more groups would 
have reduced the groups to extremely small numbers. 

Table 1 shows the means, standard deviations, and orthogonal contrast 
of the total amount of time^on-task needed by the three different groups 
of students to reach the criterion level in units one, tv/o, and three. 
The contrasts of^interest are: Group 1 versus Group 2, since both groups 
required no additional elapsed time to attain the criterion; and the 
combined group (Group 1 plus Group 2) contrasted with Group 3, si'ce Group 
3 was the only group that required additional elapsed time to attain the 
criterion on that particular' unit. 

Insert Table 1 About Here 
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An examination of the means of the first unit shows that the two 
groups that attained the criterion level in the original amount of elapsed 
time are quite similar in the amount of time-on-task required. The 
students in the 85ML class who wer^ able to attain mastery only with 
the allottment of additional amounts of elapsed time and help required 
approximately 66 per cent more on-task time on the average than did their 
classmates who were able to attain mastery in the original amount of 
elapsed time. An examination of the contrasts shows that there was no 
significant difference between the two "original elapsed time" groups* 
There was a significant difference between the combined group and the 
"additional elapsed time" group (p < .001). 

In the second unit a similar pattern is observed although there 
is one important difference. Once again there was no significant 
difference between the two "original elapsed time" groups. However, 
while the difference between the combined group and the "additional 
elapsed time" group was still significant, it was less marked than in 
unit one. The "additional elapsed time" group required approximately 
30 per cent more time-on-task on the average than their peers in the 85ML 
cl ass . , 

in the third unit this trend continues. There was no significant 
difference among the three groups. The "additional elapsed time" group 
did not differ significantly from the combined group on the amount of 
on-task time-to-criterion* The "additional elapsed time" group required 
only approximately five per cent more time-on-task than did their 85ML 
classmates. 
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As added support for the above finding it can be noted that the 
standard deviations of the two subgroups of che feBML class were quit<^ 
i^imilar* Thfr distributions of the necessary on-task time-to- criterion 
of the tfio subgroups were almost identical. 

A second way of examining the hypothesis is to examine the 
individual differences within the 85MI class in the amount of elapsed 
tin!8 and time-on-task to reach criterion over the three-task sequence. 

Table 2 shows the range of elapsed time and time-on-t^sk-to- 
criteiion for the 29 students in the 85ML class. The range was then 
converted into a ratio of fastest student *5 time-to-criterion to the 
slowest student *s time-to-criterion. 

Insert Table 2 About Here 

There was a fairly large discrepancy between the ratio of elapsed 
time-to-criterion and the ratio- of time-on-task-to-criterion in the 
first unit. Wh11e.it appeared that the slowest student needed approximately 
three and one-half times as much "time" to attain the criterion when 
elapsed time was used as the time measure, it appeared tJiat the slowest 
student required only twice as much "time" when time-cn-task was the 
"time" indicator. 

Also, the ratios o"" both elapsed and on-task time-to-criterion decreased 
as the students progressed from unit to unit. In unit three it can be 
seen that while the slowest student stlH needed twice as much elapsed tlijne 
as the fastest student, he needed only one and two-fifths as much Izime-on- 
task. 

14 
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The above findings lend much support to the hypothesis. The 
85 per cent criterion mastery learning strar^^gy was successful in 
producing students who were quite similar in the amount of on-task time 
necessary to reach the criterion on the final task. 
Conclusions and Implications 

Two major conclusions can be derived from this study. First, the 
amount of necessar/^tlme-on-task-to-criterion can be altered by an effective 
learning strategy. Second! students with varying ainounts of general 
ability can become quite similar, in the amount of time-on-task they require 
to learn a particular learning task after a series of preparatory tasks* 

This would imply that if jeguality of leaniing outcofnes is a desired 
, goal in certain instances in education, it can be achieved by designing 
learning situations that allow forjnequalities in the characteristics 
which the students bring to the task^ In other words, if, as is true in 
virtually all learning situations, students enter with unequal characteristics 
and are presented with a learning situation in which all are given ^n equal- 
amount of elapsed time and instructional help, -the students will spend 
differing amounts of their time on-task, and the result will be students 
with unequal leaving characteristics. One might recognize this as the " 
basic argument of mastery learning advocates since Bloom's (1968) original 
paper. 

The present stu^y, however, has added an important dimension to the 
previous argument* By complementing inequality in learner characteristics 
with inequality in instructional time and help in early units, we can 
approach student equality in later units, not only in the achievement 
level attained, but also in the amount o^ on-task time needed to attain 
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the criterion level. 

Thus, over a series of sequential school learning units, the argument 
is as follows. Students enter a particular learning sequence with unequal 
characteristics. Some of these characteristics are relevant to the 
learning sequence and some are not. Unequal amounts of tiire and help 
are provided for students to learn 'each unit to a relatively high leveU 
After a number of units students will approach equality in the amount of 
time-on-task they need to spepd to reach the criterion level in successive 
units. Further, and in a more practical vein, teachers will need to allot 
less and less additional elapsed time since students will spend a greater 
per cent of their tim on-task (Anderson. 1974). 

Two points are important to make in conclusion. Rirst, one cannot 
look for equality in achievement or necessary on-task time on the early, 
or preparatoT7, learning tasks since the students will enter these with 
a variety of characteristics which are relevant to the particular learning 
sequence. It is only after students have been provided with the necessary 
entering characteristics over several tasks in preparation for the later 
tasks or units that one can judge whether or not there is equality of 
learning both in time-to-criterion and the criterion level attained. 

Finally, it is not possible to judge equality of learning time-to- 
criterion with measures of elapsed time. It is only a fair comparison to 
mafce when students have spent equal amounts of time on-task, not equal 
hours of elapsed time in the classroom. 
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Table .1 



Means, Standard Deviations, and Orthogonal Contrasts of the Amount of 
Time-on-Task Needed by Three Different Groups on Three Successive 
Units to Reach the 80% Criterion Level 



Contrasts and Significance 
Gf^up N Mean Stand. Dev. Gl - 62 (Gl + G2)/2 - G3 

Unit 1 



1 : Students from 
control classes 


9 


9.71 


1.79 • 


1 .53ns 


-4.57*** 


2: Ss from 85ML class 
who reached level in 
original elapsed time 


6 


8.18 


0.47 






3: Ss from 85ML class 
who needed additional 
elapsed time 


23 


13.69 


2.01 






Unit 2 


1: Ss from control 
classes 


10 


12.04 


3.78 


1 .57ns 


-2.53* 


Of 

2: Ss from 85ML class 
who reached level in 
original elapsed time 


7 


10.47 


0.74 






3: Ss from 85ML class 
who needed additional 
elapsed time 


22 


13.61 


1.85 






Uni t 3 


1 : Ss from control 
classes 


15 


17.55 


2.57 


1.03ns 


-1.23ns 


2: Ss from 85.ML class 
who reached level in 
original elapsed time 


13 


18.58 


1.61 






3: Ss from 85ML class 
who needed additional 
elapsed time 


16 


19.30 


1.43 







Note. — The following convention for level significance is used: 
.05 level = *; .01 level = **; and .001 level = ***. 



Table 2 

Range of Elapsed Time and Time-on-Task to Criterion 
in the 85ML Class 



Time Measure 


Range of Minutes to Criterion 


Rati 0 


Unit 1 


Elapsed Time 


9 to 31 


1 to 3.4 


Time-on-Task 


7,8 to 16.5 


1 to 2.1 


Unit 2 


Elapsed Time 


10 to 28 


/I to 2.8 


Time-on-Task 


9.7 to 17.4 


1 to 1.9 


Unit 3 


Elapsed -T.i me 


16 to 33 


1 to 2.1 


Time-on-Task 


16.0 to 22.3 


1 to 1.4 



